Many important pattern classification problems remain unsolved. This category of problems includes automatic target recognition, sonar signature classification, automated fault diagnosis, natural language understanding, and robotic vision. Traditional approaches have failed due to the extreme complexity of the algorithms needed, making software development and maintenance costs unacceptably high. In addition, processing requirements for many applications are extremely large, again resulting in unacceptably high costs. Knowledge-based systems have had limited success in a few areas, such as fault diagnosis, but fall short in many of the more difficult cases. Performance and software development costs are also an issue with knowledge-based systems.
One of the reasons that many pattern classification problems are difficult to solve on a computer is that there is a basic incompatibility between the problem and the machine architecture. Modern computer systems rely on quick processors that perform operations sequentially on individual pieces of data, but this method is inappropriate when applied to some problems. For example, when viewing a picture, it is desirable to interpret it as a whole, rather than as a large number of individual picture elements (pixels) that somehow interrelate. But in order to get a sequential processor to interpret a picture, one must devise an algorithm for performing the interpretation by sequential processing of individual pixels. This has proven to be extremely difficult.
Artificial neural networks are a promising new technology that may provide a solution to these problems. Neural networks can treat complex patterns as a whole, thereby circumventing the problems associated with the traditional approach. Moreover, the ability of these networks to discover relationships in the data on their own through a training process lowers system development costs substantially. Although neural networks are able to capitalize on the inherent parallelism present in these problems, most neural network development work is done using simulations of networks on standard, single-CPU computer architectures. This tends to exacerbate the performance issue due to the computationally intensive simulations.
An introductory article describing neural networks is Learner, E. J., "Computers that learn", Aerospace America, June 1988, pp. 32 et seq.
There is a growing need for hardware and software systems designed specifically to take advantage of the massive parallelism of neural networks in order to provide sufficient computational power to meet the requirements of future applications. This must be balanced against the requirements that such hardware be cost effective, easy to use, and readily available. The present invention offers an elegant solution to this need.
Due to the universal availability of traditional sequential computer systems, it is not surprising that most neutral network development has been done directly on such machines. This requires that the inherent parallelism of the networks be software simulated, treating the single sequential processor as a virtual parallel system. Unfortunately, even modest neural network applications can overwhelm large computer systems. This is a result of the need to perform massive numbers of floating point multiply/accumulate calculations.
Hosting the simulations on larger, more powerful computers and supercomputers is one approach. However, this is not cost effective in most cases. There are also a growing number of parallel-CPU computer systems on the market which can be used to increase performance. Many of these systems rely on an arrangement such as the hypercube to facilitate communication among the processors in the system. As with the larger supercomputers, the cost of these systems is prohibitive in many cases, and hence, their availability is limited. Additionally, the design of these systems has not been optimized for neural network applications, and they do not come with special purpose software for neural networks.
A dedicated coprocessor system, tailored for neural networks, but housed in a traditional platform such as a desktop microcomputer, is an approach that has been taken by two companies, Hecht-Nielsen Neurocomputer Corporation and Science Applications International Corporation. Both vendors provide coprocessor boards for IBM PC/AT-compatable computers and specialized software for applications development, as evidenced by the specification sheets from these two companies. However, unlike the present invention, neither of these systems is a true parallel processor, and neither system is designed specifically to take advantage of the parallel computations in neural networks.
U.S. Pat. No. 4,414,624 discloses a multiple parallel processor computer system, including a dual bus structure and a combination dedicated/shared memory arrangement. The present invention offers the following advantages over the reference device: (1) means for implementing a neural network on the architecture; (2) means for transferring information from individual processors 10 to the global memory 55,56 using two busses 30,59 simultaneously; (3) means for broadcasting data from the global memory 55, 56 to all of the individual processors 10 simultaneously; (4) when one of the processors 10 sends information to the global memory 55,56, means for monitoring this information by all of the other processors 10 simultaneously, thus speeding computation for the preferred neural network application. In the reference device, on the other hand, when a processor communicates with the shared memory, the other processors do not monitor the information.
Other references are U.S. Pat. Nos. 4,214,305; 4,219,873; 4,253,146; 4,351,025; 4,486,834; 4,591,977; and 4,644,461.